Bisimulations and Logical Characterizations on Continuous-Time Markov Decision Processes
نویسندگان
چکیده
In this paper we study strong and weak bisimulation equivalences for continuous-time Markov decision processes (CTMDPs) and the logical characterizations of these relations with respect to the continuous-time stochastic logic (CSL). For strong bisimulation, it is well known that it is strictly finer than CSL equivalence. In this paper we propose strong and weak bisimulations for CTMDPs and show that for a subclass of CTMDPs, strong and weak bisimulations are both sound and complete with respect to the equivalences induced by CSL and the sub-logic of CSL without next operator respectively. We then consider a standard extension of CSL, and show that it and its sub-logic without X can be fully characterized by strong and weak bisimulations respectively over arbitrary CTMDPs.
منابع مشابه
On $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov processes
In the present paper we investigate the $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov processes with general state spaces. We provide a necessary and sufficient condition for such processes to satisfy the $L_1$-weak ergodicity. Moreover, we apply the obtained results to establish $L_1$-weak ergodicity of quadratic stochastic processes.
متن کاملBelief Bisimulation for Hidden Markov Models - Logical Characterisation and Decision Algorithm
This paper establishes connections between logical equivalences and bisimulation relations for hidden Markov models (HMM). Both standard and belief state bisimulations are considered. We also present decision algorithms for the bisimilarities. For standard bisimilarity, an extension of the usual partition refinement algorithm is enough. Belief bisimilarity, being a relation on the continuous sp...
متن کاملThe Inventory System Management under Uncertain Conditions and Time Value of Money
This study develops a inventory model to determine ordering policy for deteriorating items with shortages under markovian inflationary conditions. Markov processes include process whose future behavior cannot be accurately predicted from its past behavior (except the current or present behavior) and which involves random chance or probability. Behavior of business or economy, flow of traffic, p...
متن کاملBisimulation and Logical Preservation for Continuous-Time Markov Decision Processes
This paper introduces strong bisimulation for continuous-timeMarkov decision processes (CTMDPs), a stochastic model which allows for a nondeterministic choice between exponential distributions, and shows that bisimulation preserves the validity of CSL. To that end, we interpret the semantics of CSL—a stochastic variant of CTL for continuous-time Markov chains—on CTMDPs and show its measure-theo...
متن کاملContinuous-time Markov decision processes with nth-bias optimality criteria
In this paper, we study the nth-bias optimality problem for finite continuous-time Markov decision processes (MDPs) with a multichain structure. We first provide nth-bias difference formulas for two policies and present some interesting characterizations of an nth-bias optimal policy by using these difference formulas. Then, we prove the existence of an nth-bias optimal policy by using nth-bias...
متن کامل